Systems and methods for brain hemorrhage classification in medical images using an artificial intelligence network

ABSTRACT

Systems and methods for rapid, accurate, fully-automated, brain hemorrhage deep learning (DL) based assessment tools are provided, to assist clinicians in the detection &amp; characterization of hemorrhages or bleeds. Images may be acquired from a subject using an imaging source, and preprocessed to cleanup, reformat, and perform any needed interpolation prior to being analyzed by an artificial intelligence network, such as a convolutional neural network (CNN). The artificial intelligence network identifies and labels regions of interest in the image, such as identifying any hemorrhages or bleeds. An output for a user may also include a confidence value associated with the identification.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional PatentApplication Ser. No. 62/555,783, filed on Sep. 8, 2017, and entitled “AMETHODOLOGY FOR RAPID, ACCURATE, AND SCRUTINIZABLE BRAIN HEMORRHAGECLASSIFICATION ARTIFICIAL INTELLIGENCE.”

BACKGROUND

Deep learning has demonstrated significant success in improvingdiagnostic accuracy, speed of image interpretation, and clinicalefficiency for a wide range of medical tasks ranging from interstitialpattern detection on chest CT to bone age classification on handradiographs. Particularly, a data-driven approach with deep neuralnetworks has been actively utilized for several medical imagesegmentation applications, including segmenting brain tumors on magneticresonance images, segmenting organs of interest on CT, and segmentingthe vascular network of the human eye on fundus photography. Thesesuccesses are attributed to the capability of deep learning to learnrepresentative and hierarchical image features from data, rather thanrelying on manually engineered features based on knowledge from domainexperts.

However, many deep learning networks are unable to process images at aspeed that meets clinical needs and may not be sufficiently accurate towarrant being relied upon in a clinical setting. In still othersettings, deep learning routines may not be sensitive enough todistinguish between regions in an image.

SUMMARY OF THE DISCLOSURE

The present disclosure addresses the aforementioned drawbacks byproviding a method for identifying a condition in a medical image of asubject. The method includes accessing a medical image of a subject witha computer system. A region-of-interest (ROI) is identified in themedical imaging by processing the medical image with an artificialintelligence (AI) network implemented with a hardware processor and amemory of the computer system. The ROI is labeled with an identificationof a first brain hemorrhage condition in the ROI using an output of theAI network. A report is generated for a user identifying the first brainhemorrhage condition and including a confidence value associated with anidentification of the first brain hemorrhage condition in the ROI.

It is another aspect of the present disclosure to provide a system foridentifying a brain hemorrhage condition in an image of a subject. Thesystem includes at least one hardware processor and a memory. The memoryhas stored thereon instructions that when executed by the at least onehardware processor cause the at least one hardware processor to performsteps including: (a) accessing a medical image of a subject from thememory; (b) accessing a trained convolutional neural network (CNN) fromthe memory, the trained CNN having been trained on medical imageslabeled with one or more different brain hemorrhage conditions; (c)generating a class activation map that indicates at least one brainhemorrhage condition in the medical image by processing the medicalimage with the AI network; (d) generating a labeled image by labellingregions in the medical image associated with the at least one brainhemorrhage condition using the class activation map; (e) producing atleast one confidence value for each labeled region in the labeled image,each confidence value being associated with a confidence of each labeledregion representing a corresponding at least one brain hemorrhagecondition; and (f) generating a display that depicts the labeled imageand the at least one confidence value.

The foregoing and other aspects and advantages of the present disclosurewill appear from the following description. In the description,reference is made to the accompanying drawings that form a part hereof,and in which there is shown by way of illustration a preferredembodiment. This embodiment does not necessarily represent the fullscope of the invention, however, and reference is therefore made to theclaims and herein for interpreting the scope of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A and 1B illustrate an example CT system that can be configuredto operate some configurations of the present disclosure.

FIG. 2 is a schematic for one configuration of the present disclosureusing a computer system.

FIG. 3 is another schematic for one configuration of the presentdisclosure using a computer system.

FIG. 4 is a flowchart depicting one configuration of the disclosure foridentifying hemorrhages or bleeds using a convolutional neural network(CNN).

FIG. 5 is a schematic for one configuration of window leveling anoriginal image into multiple color channels that are then combined forcreating a reformatted image.

FIG. 6 is a depiction of image slice interpolation that may be used inone configuration of the present disclosure.

FIG. 7A is a graph representation of the results of an example binaryclassification scheme using a receiver operating characteristics (ROC)curve analysis.

FIG. 7B is a graph representation of the results of an examplemulti-label classification approach according to one configuration ofthe disclosure using a receiver operating characteristics (ROC) curveanalysis.

FIG. 8 is a depiction of an example output report for one configurationshowing the results of a confidence analysis.

FIG. 9 is a depiction of an example output report for one configurationinvolving multiple hemorrhages identified by slice location.

DETAILED DESCRIPTION

Systems and methods for rapid, accurate, fully-automated brainhemorrhage deep learning (DL) based assessment tools are provided. Thesesystems and methods can be used, as an example, to assist clinicians inthe detection and characterization of hemorrhages. Hemorrhages mayinclude intracranial hemorrhage (ICH), subarachnoid hemorrhage (SAH),intra-parenchymal hemorrhage (IPH), epidural/subdural hematoma (SDH),and the like.

Referring particularly now to FIGS. 1A and 1B, an example of an x-raycomputed tomography (“CT”) imaging system 100 is illustrated for usewith some configurations of the disclosure. The CT system includes agantry 102, to which at least one x-ray source 104 is coupled. The x-raysource 104 projects an x-ray beam 106, which may be a fan-beam orcone-beam of x-rays, towards a detector array 108 on the opposite sideof the gantry 102. The detector array 108 includes a number of x-raydetector elements 110. Together, the x-ray detector elements 110 sensethe projected x-rays 106 that pass through a subject 112, such as amedical patient or an object undergoing examination, that is positionedin the CT system 100. Each x-ray detector element 110 produces anelectrical signal that may represent the intensity of an impinging x-raybeam and, hence, the attenuation of the beam as it passes through thesubject 112. In some configurations, each x-ray detector 110 is capableof counting the number of x-ray photons that impinge upon the detector110. In some configurations the system can include a second x-ray sourceand a second x-ray detector (not shown) operable at a different energylevel than x-ray source 104 and detector 110. Any number of x-raysources and corresponding x-ray detectors operable at different energiesmay be used, or a single x-ray source 104 may be operable to emitdifferent energies that impinge upon detector 110. During a scan toacquire x-ray projection data, the gantry 102 and the components mountedthereon rotate about a center of rotation 114 located within the CTsystem 100.

The CT system 100 also includes an operator workstation 116, whichtypically includes a display 118; one or more input devices 120, such asa keyboard and mouse; and a computer processor 122. The computerprocessor 122 may include a commercially available programmable machinerunning a commercially available operating system. The operatorworkstation 116 provides the operator interface that enables scanningcontrol parameters to be entered into the CT system 100. In general, theoperator workstation 116 is in communication with a data store server124 and an image reconstruction system 126. By way of example, theoperator workstation 116, data store sever 124, and image reconstructionsystem 126 may be connected via a communication system 128, which mayinclude any suitable network connection, whether wired, wireless, or acombination of both. As an example, the communication system 128 mayinclude both proprietary or dedicated networks, as well as opennetworks, such as the internet.

The operator workstation 116 is also in communication with a controlsystem 130 that controls operation of the CT system 100. The controlsystem 130 generally includes an x-ray controller 132, a tablecontroller 134, a gantry controller 136, and a data acquisition system138. The x-ray controller 132 provides power and timing signals to thex-ray source 104 and the gantry controller 136 controls the rotationalspeed and position of the gantry 102. The table controller 134 controlsa table 140 to position the subject 112 in the gantry 102 of the CTsystem 100.

The DAS 138 samples data from the detector elements 110 and converts thedata to digital signals for subsequent processing. For instance,digitized x-ray data is communicated from the DAS 138 to the data storeserver 124. The image reconstruction system 126 then retrieves the x-raydata from the data store server 124 and reconstructs an image therefrom.The image reconstruction system 126 may include a commercially availablecomputer processor, or may be a highly parallel computer architecture,such as a system that includes multiple-core processors and massivelyparallel, high-density computing devices. Optionally, imagereconstruction can also be performed on the processor 122 in theoperator workstation 116. Reconstructed images can then be communicatedback to the data store server 124 for storage or to the operatorworkstation 116 to be displayed to the operator or clinician.

The CT system 100 may also include one or more networked workstations142. By way of example, a networked workstation 142 may include adisplay 144; one or more input devices 146, such as a keyboard andmouse; and a processor 148. The networked workstation 142 may be locatedwithin the same facility as the operator workstation 116, or in adifferent facility, such as a different healthcare institution orclinic.

The networked workstation 142, whether within the same facility or in adifferent facility as the operator workstation 116, may gain remoteaccess to the data store server 124 and/or the image reconstructionsystem 126 via the communication system 128. Accordingly, multiplenetworked workstations 142 may have access to the data store server 124and/or image reconstruction system 126. In this manner, x-ray data,reconstructed images, or other data may be exchanged between the datastore server 124, the image reconstruction system 126, and the networkedworkstations 142, such that the data or images may be remotely processedby a networked workstation 142. This data may be exchanged in anysuitable format, such as in accordance with the transmission controlprotocol (“TCP”), the internet protocol (“IP”), or other known orsuitable protocols.

Referring to FIG. 2, an example 200 of a system for automaticallydetecting a hemorrhage or a bleed using image data in accordance withsome configurations of the disclosed subject matter is shown. Acomputing device 210 can receive multiple types of image data from imagesource 202. In some configurations, image source 202 may be a CT system.In some embodiments, computing device 210 can execute at least a portionof an automatic identification system 204 to automatically determinewhether a patient condition, such as a hemorrhage, is present in imagesof a subject.

Additionally or alternatively, in some embodiments, computing device 210can communicate information about image data received from image source202 to a server 220 over a communication network 208, which can executeat least a portion of automatic identification system 204. In suchembodiments, server 220 can return information to computing device 210(and/or any other suitable computing device) indicative of an output ofautomatic identification system 204 to determine whether a clinicalcondition is present or absent.

In some embodiments, computing device 210 and/or server 220 can be anysuitable computing device or combination of devices, such as a desktopcomputer, a laptop computer, a smartphone, a tablet computer, a wearablecomputer, a server computer, a virtual machine being executed by aphysical computing device, etc. In some embodiments, automaticidentification system 204 can extract features from labeled (e.g.,labeled as including diseased or normal) image data, such as CT imagedata, using a convolutional neural network (CNN) trained as a generalimage classifier.

In some embodiments, image source 202 can be any suitable source ofimage data, such as a CT machine, another computing device (e.g., aserver storing CT image data), etc. In some embodiments, image source202 can be local to computing device 210. For example, image source 202can be incorporated with computing device 210 (e.g., computing device210 can be configured as part of a device for capturing and/or storingimages). As another example, image source 202 can be connected tocomputing device 210 by a cable, a direct wireless link, etc.Additionally or alternatively, in some embodiments, image source 202 canbe located locally and/or remotely from computing device 210, and cancommunicate image data to computing device 210 (and/or server 220) via acommunication network (e.g., communication network 208).

In some embodiments, communication network 208 can be any suitablecommunication network or combination of communication networks. Forexample, communication network 208 can include a Wi-Fi network (whichcan include one or more wireless routers, one or more switches, etc.), apeer-to-peer network (e.g., a Bluetooth network), a cellular network(e.g., a 3G network, a 4G network, etc., complying with any suitablestandard, such as CDMA, GSM, LTE, LTE Advanced, WiMAX, etc.), a wirednetwork, etc. In some embodiments, communication network 208 can be alocal area network, a wide area network, a public network (e.g., theInternet), a private or semi-private network (e.g., a corporate oruniversity intranet), any other suitable type of network, or anysuitable combination of networks. Communications links shown in FIG. 2can each be any suitable communications link or combination ofcommunications links, such as wired links, fiber optic links, Wi-Filinks, Bluetooth links, cellular links, etc.

FIG. 3 shows an example 300 of hardware that can be used to implementimage source 202, computing device 210, and/or server 220 in accordancewith some embodiments of the disclosed subject matter. As shown in FIG.3, in some embodiments, computing device 210 can include a processor302, a display 304, one or more inputs 306, one or more communicationsystems 308, and/or memory 310. In some embodiments, processor 302 canbe any suitable hardware processor or combination of processors, such asa central processing unit (CPU), a graphics processing unit (GPU), etc.In some embodiments, display 304 can include any suitable displaydevices, such as a computer monitor, a touchscreen, a television, etc.In some embodiments, inputs 306 can include any suitable input devicesand/or sensors that can be used to receive user input, such as akeyboard, a mouse, a touchscreen, a microphone, etc.

In some embodiments, communications systems 308 can include any suitablehardware, firmware, and/or software for communicating information overcommunication network 208 and/or any other suitable communicationnetworks. For example, communications systems 308 can include one ormore transceivers, one or more communication chips and/or chip sets,etc. In a more particular example, communications systems 308 caninclude hardware, firmware and/or software that can be used to establisha Wi-Fi connection, a Bluetooth connection, a cellular connection, anEthernet connection, etc.

In some embodiments, memory 310 can include any suitable storage deviceor devices that can be used to store instructions, values, etc., thatcan be used, for example, by processor 302 to present content usingdisplay 304, to communicate with server 220 via communications system(s)308, etc. Memory 310 can include any suitable volatile memory,non-volatile memory, storage, or any suitable combination thereof. Forexample, memory 310 can include RAM, ROM, EEPROM, one or more flashdrives, one or more hard disks, one or more solid state drives, one ormore optical drives, etc. In some embodiments, memory 310 can haveencoded thereon a computer program for controlling operation ofcomputing device 210. In such embodiments, processor 302 can execute atleast a portion of the computer program to present content (e.g., CTimages, user interfaces, graphics, tables, etc.), receive content fromserver 220, transmit information to server 220, etc.

In some embodiments, server 220 can include a processor 312, a display314, one or more inputs 316, one or more communications systems 318,and/or memory 320. In some embodiments, processor 312 can be anysuitable hardware processor or combination of processors, such as a CPU,a GPU, etc. In some embodiments, display 314 can include any suitabledisplay devices, such as a computer monitor, a touchscreen, atelevision, etc. In some embodiments, inputs 316 can include anysuitable input devices and/or sensors that can be used to receive userinput, such as a keyboard, a mouse, a touchscreen, a microphone, etc.

In some embodiments, communications systems 318 can include any suitablehardware, firmware, and/or software for communicating information overcommunication network 208 and/or any other suitable communicationnetworks. For example, communications systems 318 can include one ormore transceivers, one or more communication chips and/or chip sets,etc. In a more particular example, communications systems 318 caninclude hardware, firmware and/or software that can be used to establisha Wi-Fi connection, a Bluetooth connection, a cellular connection, anEthernet connection, etc.

In some embodiments, memory 320 can include any suitable storage deviceor devices that can be used to store instructions, values, etc., thatcan be used, for example, by processor 312 to present content usingdisplay 314, to communicate with one or more computing devices 210, etc.Memory 320 can include any suitable volatile memory, non-volatilememory, storage, or any suitable combination thereof. For example,memory 320 can include RAM, ROM, EEPROM, one or more flash drives, oneor more hard disks, one or more solid state drives, one or more opticaldrives, etc. In some embodiments, memory 320 can have encoded thereon aserver program for controlling operation of server 220. In suchembodiments, processor 312 can execute at least a portion of the serverprogram to transmit information and/or content (e.g., CT data, resultsof automatic identification, a user interface, etc.) to one or morecomputing devices 210, receive information and/or content from one ormore computing devices 210, receive instructions from one or moredevices (e.g., a personal computer, a laptop computer, a tabletcomputer, a smartphone, etc.), etc.

In some embodiments, image source 202 can include a processor 322,imaging components 324, one or more communications systems 326, and/ormemory 328. In some embodiments, processor 322 can be any suitablehardware processor or combination of processors, such as a CPU, a GPU,etc. In some embodiments, imaging components 324 can be any suitablecomponents to generate image data. An example of an imaging machine thatcan be used to implement image source 202 can include a conventional CTscanner and the like.

Note that, although not shown, image source 202 can include any suitableinputs and/or outputs. For example, image source 202 can include inputdevices and/or sensors that can be used to receive user input, such as akeyboard, a mouse, a touchscreen, a microphone, a trackpad, a trackball,hardware buttons, software buttons, etc. As another example, imagesource 202 can include any suitable display devices, such as a computermonitor, a touchscreen, a television, etc., one or more speakers, etc.

In some embodiments, communications systems 326 can include any suitablehardware, firmware, and/or software for communicating information tocomputing device 210 (and, in some embodiments, over communicationnetwork 208 and/or any other suitable communication networks). Forexample, communications systems 326 can include one or moretransceivers, one or more communication chips and/or chip sets, etc. Ina more particular example, communications systems 326 can includehardware, firmware and/or software that can be used to establish a wiredconnection using any suitable port and/or communication standard (e.g.,VGA, DVI video, USB, RS-232, etc.), Wi-Fi connection, a Bluetoothconnection, a cellular connection, an Ethernet connection, etc.

In some embodiments, memory 328 can include any suitable storage deviceor devices that can be used to store instructions, values, image data,etc., that can be used, for example, by processor 322 to: controlimaging components 324, and/or receive image data from imagingcomponents 324; generate images; present content (e.g., CT images, auser interface, etc.) using a display; communicate with one or morecomputing devices 610; etc. Memory 328 can include any suitable volatilememory, non-volatile memory, storage, or any suitable combinationthereof. For example, memory 328 can include RAM, ROM, EEPROM, one ormore flash drives, one or more hard disks, one or more solid statedrives, one or more optical drives, etc. In some embodiments, memory 328can have encoded thereon a program for controlling operation of imagesource 202. In such embodiments, processor 322 can execute at least aportion of the program to generate images, transmit information and/orcontent (e.g., CT image data) to one or more computing devices 210,receive information and/or content from one or more computing devices210, receive instructions from one or more devices (e.g., a personalcomputer, a laptop computer, a tablet computer, a smartphone, etc.),etc.

Referring to FIG. 4, a flowchart is shown for one configuration of thedisclosure. At step 410, images acquired from a subject using an imagingsource, such as a CT system, are accessed. Accessing the images caninclude acquiring the images with the imaging source, or can includeaccessing or otherwise retrieving previously acquired images that arestored in a PACS, a memory, or another suitable data storage. Theaccessed image can include an image volume. The image volume may includea three-dimensional image volume containing a plurality of spatiallyadjacent image slices. The image volume may also include a series oftemporally adjacent images, such as those acquired as a time series ofimages from one or more image slice locations. The images may be imagesacquired with a CT system, an MRI system, or any other suitable medicalimaging system.

The images may be preprocessed, as indicated at process block 420.Preprocessing of the acquired images may include cleanup of the imagesat step 430, reformatting of the images at step 440, and interpolationat step 450. Image cleanup at step 430 may include applying masks to theimage to localize specific regions of interest for the CNN to process,and may include removing background signal from the images.

Referring to FIG. 5, the image reformatting performed at step 440 inFIG. 4 may include applying different window/level settings to the inputimage 510 in order to generate multiple different channel images 512.Each channel image can be generated by applying a specified window/levelsetting to pixels in the input image 510. For instance, a specifiedwindow/level setting can be applied to pixels having intensity values ina specified range associated with the specified window. As anotherexample, a specified window/level setting can be applied to pixelshaving quantitative values, such as Hounsfield Units (HU), within aspecified range associated with the specified window. Any number ofchannel images may be generated, and a reformatted image 550 can becreated by combining the channel images.

In some implementations, the different channel images may be colorized,such as mapping pixel intensity values in the channel images to one ormore different color scales, or by otherwise assigning a specific colorto each different channel image. For example, a red channel image 520may be generated using a window/level setting with a level WL=60 andwindow WW=40 for pixels in the input image 510 corresponding to HUvalues in the range of 40-80. The pixel values in the red channel image520 are then assigned a suitable RGB value, such as by mapping the pixelvalues to an RGB color scale. A green channel image 530 may be generatedusing a window/level setting with a level WL=50 and a window WW=100 forpixels in the input image 510 corresponding to HU values in the range of0-100. The pixel values in the green channel image 530 are then assigneda suitable RGB value, such as by mapping the pixel values to an RGBcolor scale. A blue channel image 540 may be generated using awindow/level setting with a level WL=40 and a window WW=40 for pixels inthe input image 510 corresponding to HU values in the range of 20-60.The pixel values in the blue channel image 540 are then assigned asuitable RGB value, such as by mapping the pixel values to an RGB colorscale. When the different channel images are assigned different colors(e.g., by converting grayscale values to RGB values, or values from adifferent colormap or color scale), the reformatted image 550 may bestored or presented to a user as a multi-color image. In some instances,the channel images can be combined to form a combined image (e.g., anRGB image when combining red, green, and blue channel images).

Image interpolation performed at step 450 of FIG. 4 may be performed tomodel data between slices. This form of interpolation may also be usedwhen creating 3D models from a series of 2D images where there is adesire to increase the resolution of the series of 2D images by reducingthe space between slices. A new interpolated slice may be created asfollows:

S _(interpolation) =σS ₁+(1−α)S ₂  (1);

where (0≤α≤1), α is a mixing weight that may be based on the number ofdesired interpolated slices and the proximity of the created slice inthe stack to either S₁ or S₂; S₁ is a first original slice, S₂ is asecond original slice, and S_(interpolation) is the interpolated slicesituated between S₁ and S₂. Referring to FIG. 6, an example is shownwhere six slices are desired between a first original slice S₁₂ and asecond original slice S₁₃ where α may be 0.80, 0.67, 0.60, 0.40, 0.33,and 0.20 for each of the six slices shown between S₁₂ and S₁₃. In someimplementations, the same mixing weight can be used for eachinterpolation; however, in some other implementations one or moredifferent mixing weights can be used for different interpolation steps.For instance, a first mixing weight may be used when interpolatingbetween a first and second slice, but a second mixing weight that isdifferent from the first may be used when interpolating between thesecond slice and a third slice.

Referring again to FIG. 4, the processed images are then input to andanalyzed by the artificial intelligence (AI) network, which may be aconvolutional neural network (CNN), at step 460. The AI network isgenerally implemented with one or more hardware processors and one ormore memories. In some implementations, real-time data augmentation maybe performed on the processed images before they are input into the AInetwork. Real-time augmentation may include horizontal flipping,rescaling, rotation, denoising, and the like. The AI network may bepreviously trained using labelled or annotated images prior to analyzingimages acquired from the subject. Labelled or annotated images mayinclude, for instance, images in which regions-of-interest have beenidentified and to which a label or annotation has been assigned. In someinstances, a labelled or annotated image can include an image that hasbeen suitably modified to include such labels or annotations. As oneexample, the labelled or annotated regions could be color coded. In someother instances, the labelled or annotated images could include aseparate image or map that depicts the corresponding locations of thelabelled or annotated regions. As one example, the separate image or mapcould be a class activation map, such as those generated whenimplementing the systems and methods described in the presentdisclosure. As noted, in some embodiments, the AI network can be a CNN.The CNN could be implemented using an ImageNet CNN, a GoogleNet CNN, aVGG16 CNN, and so on.

Class activation maps are generated at step 470 as an output from the AInetwork and are used to identify and label different regions or objectsof interest in the images at step 480. The identifying and labelingprocess may include labeling hemorrhages, such as IPH, ICH, IVH, SDH,and SAH. A bleed may also be identified and labeled. Using a “bleed”label and in some instances improve the sensitivity of the analysis. Thedifferent labels can be generated using different loss penalties toovercome class imbalance.

At step 490, a prediction is output for a user indicating the results ofthe identification and labeling processes and may include a confidencevalue associated with the identification. Confidence values may bereported between 0-100 in percent, or may be reported between 0-1, or inanother form. For example, the output may indicate the presence of a SAHin a particular region of the image with a 99% confidence value.Alternatively, the output may indicate the results and associatedconfidence values for all of the potential hemorrhages or bleeds. Forexample, the output could include IPH=1%, IVH=1%, SDH=3%, SAH=95%,bleed=0%, along with a corresponding image highlighting the one or moreregions that were identified as being of interest. A particular subjectmay suffer from multiple hemorrhages of bleeds, which may each fall intodifferent classifications.

Referring to FIGS. 7A and 7B, an example receiver operatingcharacteristics (ROC) curve is shown in FIG. 7A where true-positive rate(sensitivity) is plotted against 1-specificity for a binaryclassification method. An area under the curve (AUC) of the ROC helpsdefine the diagnostic capabilities of a method. FIG. 7B displays exampleresults of a multi-label classification as discussed in FIG. 4 with asimilar ROC curve analysis. AUC values are comparable for differenthemorrhage identifications. A similar AUC analysis of an ROC curve maybe used as an output to a user at step 490 in FIG. 4 in addition to orin place of the confident values.

In one example of the present disclosure, a retrospective study wasconducted. A PACS system was searched for unenhanced head CTs of IPH,SAH, and SDH/EDH. Two radiologists reviewed the scans and selectedslices with hemorrhage for training. The training set was enriched withcases that had subtle or small volume bleeds. 100 slices were randomlyselected from each class for validation, independent of the trainingset. Full resolution 512×512 pixel CT images were converted into graylevels by applying a brain CT window setting (window-level=50 HU,window-width=100 HU). A Transfer learning methodology was applied usinga customized (weight initialized) ImageNet pre-trained ConvolutionalNeural Network (CNN), fine-tuned on the training dataset.

A total of 1058 normal, 412 SAH, 287 IPH, 414 SDH, and 233 EDH head CTslices were collected from 36 normal, 46 SAH, 50 IPH, 25 SDH, and 25 EDHpatients. AUC for detection of hemorrhage was 0.96 for normal patients,0.90 for SAH, 0.95 for IPH, 0.89 for SDH, and 0.98 for EDH. Thedeployment time for full head CT 3D-volume assessment was taken lessthan 2-sec (mean) for all slices. These results reflect a high accuracyfor the detection and characterization of intracranial hemorrhage.Results for the example study are shown in Table 1.

TABLE 1 Accession Numbers Patients Axial Slices Propor- Propor- Propor-Label Count tion (%) Count tion (%) Count tion (%) IPH 401 47 280 382750 23 IVH 184 22 140 19 1571 13 SDH 47 6 43 6 401 3 EDH 1 0 1 0 4 0SAH 269 32 248 34 1350 11 BGC 91 11 88 12 218 2 No Bleed 206 24 206 287044 60 Bleed 578 68 455 62 4606 39 NoBleed 275 32 275 38 7212 61 Total853 100 730 100 11818 100

Data was split in the example study among a training set, a validationset, and a test set. The data split for the training set is reflected inTable 2. The data split for the validation set is reflected in Table 3.The data set for the test set is reflected in Table 4.

TABLE 2 Type #PID #ACC # Slices Bleed IPH 218 339 2251 IVH 85 116 1250SDH 12 16 225 SAH 164 176 1021 Total 354 477 3682 No Bleed BGC 48 48 104None 122 122 4176 Total 170 170 4280 Total 524 647 7962

TABLE 3 Type #ACC (#PID) # Slices Bleed IPH 62 499 IVH 35 321 SDH 24 171SAH 63 329 Total 100 919 No Bleed BGC 21 64 None 79 2688 Total 100 2752Total 200 3667

TABLE 4 Type #ACC (#PID) Bleed IPH 57 IVH 59 SDH 44 SAH 74 EDH 19 Total100 No Bleed Total 100 Total 200

Referring to FIG. 8, one configuration for displaying results of ahemorrhage detection system 810 for a user is shown. Example ROC curveanalysis 820 displays the results of the above study, but is presentedhere as an example of how an ROC graph may be displayed for a user withAUC analysis report 830. The results of the study above are alsoreflected in the confusion matrix report 840. This confusion matrixrepresents the likelihood for a particular hemorrhage or condition to bereported accurately and not to be confused with a different hemorrhageor condition.

Referring to FIG. 9, one configuration for reporting the results for apatient that presents with multiple hemorrhages or conditions is shown.Original image 920 shows a patient presenting with both SAH and IPH.This image is input to the hemorrhage detection system 910 which followsthe analysis of FIG. 4. The multiple hemorrhages are identified by sliceand reported with a corresponding confidence value. For example, withthe patient with original image 920 depicted, slice 930 reports an IPHwith 100% confidence. Slice 940 reflects a mix of IPH at 54% confidenceand SAH with 46% confidence. Slice 950 reflects SAH with 100%confidence, and slice 960 reflects normal with 85% confidence and SAHwith 15% confidence. These are examples, and one skilled in the art willappreciate that different original images from different patients willpresent with different results, include a report of any of thehemorrhages or conditions as described previously.

The present disclosure has described one or more preferred embodiments,and it should be appreciated that many equivalents, alternatives,variations, and modifications, aside from those expressly stated, arepossible and within the scope of the invention.

1. A method for identifying a condition in a medical image of a subjectcomprising: a) accessing a medical image of a subject with a computersystem; b) identifying a region-of-interest (ROI) in the medical imagingby processing the medical image with an artificial intelligence (AI)network implemented with a hardware processor and a memory of thecomputer system; c) labelling the ROI with an identification of a firstbrain hemorrhage condition in the ROI using an output of the AI network;and d) generating a report for a user identifying the first brainhemorrhage condition and including a confidence value associated with anidentification of the first brain hemorrhage condition in the ROI. 2.The method of claim 1 wherein the first brain hemorrhage conditionincludes at least one of an intracranial hemorrhage (ICH), anintraventricular hemorrhage (IVH), a subarachnoid hemorrhage (SAH), anintra-parenchymal hemorrhage (IPH), an epidural hematoma (EDH), asubdural hematoma (SDH), or a bleed.
 3. The method of claim 1 furthercomprising identifying a second ROI in the medical imaging by processingthe medical image with the AI network, and labelling the second ROI withan identification of a second brain hemorrhage condition using an outputof the AI network.
 4. The method of claim 3 wherein generating thereport includes displaying a first image slice in which the first brainhemorrhage condition is identified and displaying a second image slicein which the second brain hemorrhage condition is identified.
 5. Themethod of claim 3 wherein generating the report includes displaying animage slice in which both the first brain hemorrhage condition and thesecond brain hemorrhage condition are identified, and whereinidentifying the second condition includes a second confidence valueassociated with an identification of the second brain hemorrhagecondition in the second ROI.
 6. The method of claim 1 wherein processingthe medical image with the AI network comprises: generating a pluralityof channel images from the medical image, each channel image beinggenerated by applying at least one of a window setting and a levelsetting to the medical image; and inputting the plurality of channelimages to the AI network.
 7. The method of claim 6 wherein the pluralityof channel images are combined to form a combined image and inputtingthe plurality of channel images to the AI network comprises inputtingthe combined image to the AI network.
 8. The method of claim 6 whereineach of the plurality of channel images is assigned a different color.9. The method of claim 8 wherein each of the plurality of channel imagesis assigned the different color by mapping pixel intensities in eachchannel image to a different color scale.
 10. The method of claim 9wherein the plurality of channel images comprises a first channel image,a second channel image, and a third channel image, and wherein pixelintensities in the first channel image are mapped to a red color scale,pixel intensities in the second channel image are mapped to a greencolor scale, and pixel intensities in the third channel image are mappedto a blue color scale.
 11. The method of claim 1 further comprisingpreprocessing the medical image prior to inputting the medical image tothe AI network, wherein preprocessing the medical image includesreducing noise in the medical image.
 12. The method of claim 1 whereinthe medical image is an image volume comprising a plurality of imageslices, and wherein the plurality of image slices are interpolatedbefore inputting the medical image to the AI network.
 13. The method ofclaim 12 wherein the plurality of image slices are interpolated togenerate at least one new image slice between two of the plurality ofimage slices.
 14. The method of claim 12 wherein the image volume is athree-dimensional image volume comprising a stack of spatially adjacentimage slices.
 15. The method of claim 12 wherein the image volumecomprises a series of temporally adjacent image slices.
 16. The methodof claim 1 wherein the AI network is a convolutional neural network. 17.The method of claim 1 wherein the output of the AI network comprises aclass activation map.
 18. A system for identifying a brain hemorrhagecondition in an image of a subject comprising: at least one hardwareprocessor; a memory having stored thereon instructions that whenexecuted by the at least one hardware processor cause the at least onehardware processor to perform steps comprising: a) accessing a medicalimage of a subject from the memory; b) accessing a trained convolutionalneural network (CNN) from the memory, the trained CNN having beentrained on medical images labeled with one or more different brainhemorrhage conditions; c) generating a class activation map thatindicates at least one brain hemorrhage condition in the medical imageby processing the medical image with the AI network; d) generating alabeled image by labelling regions in the medical image associated withthe at least one brain hemorrhage condition using the class activationmap; e) producing at least one confidence value for each labeled regionin the labeled image, each confidence value being associated with aconfidence of each labeled region representing a corresponding at leastone brain hemorrhage condition; and f) generating a display that depictsthe labeled image and the at least one confidence value.
 19. The systemof claim 18 wherein the at least one brain hemorrhage condition includesat least one of an intracranial hemorrhage (ICH), an intraventricularhemorrhage (IVH), a subarachnoid hemorrhage (SAH), an intra-parenchymalhemorrhage (IPH), an epidural hematoma (EDH), a subdural hematoma (SDH),or a bleed.